Summary

• Bug: In save_payload_as_json, when merge=True, the file read happens inside the FileLock but the file write happens outside it. This creates a race condition where multiple concurrent processes can read the file, then all try to write simultaneously, resulting in JSON corruption or lost data.
• Impact: This affects any environment where multiple pytest processes merge results into the same JSON file concurrently — for example, when pants runs pytest in parallel across multiple targets and each process merges its results into a shared output file.
• Fix: Move the file write inside the FileLock context manager when merge=True, ensuring the read-modify-write cycle is atomic. Non-merge writes don't need the lock and are handled in a separate else branch.

Before (buggy)

if merge:
    lock = FileLock(f"{path}.lock")
    with lock:
        if os.path.exists(path):
            with open(path, "r", encoding="utf-8") as f:
                existing_data = json.load(f)
            data = existing_data + data

with open(path, "w", encoding="utf-8") as f:  # <-- OUTSIDE the lock!
    json.dump(data, f)

After (fixed)

if merge:
    lock = FileLock(f"{path}.lock")
    with lock:
        if os.path.exists(path):
            with open(path, "r", encoding="utf-8") as f:
                existing_data = json.load(f)
            data = existing_data + data
        with open(path, "w", encoding="utf-8") as f:  # <-- INSIDE the lock
            json.dump(data, f)
else:
    with open(path, "w", encoding="utf-8") as f:
        json.dump(data, f)

Test plan

• All 5 existing save_json tests pass (test_save_json_payload_without_merge, test_save_json_payload_with_merge, test_save_json_payload_with_non_existent_file, test_save_json_payload_with_invalid_file, test_save_json_payload_with_large_data)
• Verify in a concurrent environment (e.g., pants with parallel pytest targets) that the merged JSON output is no longer corrupted